Modeling Future Cost for Neural Machine Translation

Existing neural machine translation (NMT) systems utilize sequence-to-sequence networks to generate target word by word, and then make the generated at each time-step counterpart in references as consistent possible. However, trained model tends focus on ensuring accuracy of current does not consider its future cost which means expected generating subsequent (i.e., next word). To respond this issue, article, we propose a simple effective method for NMT systems. In detail, representation is learned based contextual information compute an additional loss guide training model. Furthermore, used help generation decoding. Experimental results three widely-used datasets, including WMT14 English-to-German, English-to-French, WMT17 Chinese-to-English, show that proposed approach achieves significant improvements over strong Transformer-based baseline.